A content based method for perceptually driven joint color/depth compression

International audienceMulti-view Video plus Depth (MVD) data refer to a set of conventional color video sequences and an associated set of depth video sequences, all acquired at slightly different viewpoints. This huge amount of data necessitates a reliable compression method. However, there is no standardized compression method for MVD sequences. H.264/MVC compression method, which was standardized for Multi-View-Video representation (MVV), has been the subject of many adaptations to MVD. However, it has been shown that MVC is not well adapted to encode multi-view depth data. We propose a novel option as for compression of MVD data. Its main purpose is to preserve joint color/depth consistency. The originality of the proposed method relies on the use of the decoded color data as a prior for the associated depth compression. This is meant to ensure consistency in both types of data after decoding. Our strategy is motivated by previous studies of artifacts occurring in synthesized views: most annoying distortions are located around strong depth discontinuities and these distortions are due to misalignment of depth and color edges in decoded images. Thus the method is meant to preserve edges and to ensure consistent localization of color edges and depth edges. To ensure compatibility, colored sequences are encoded with H.264. Depth maps compression is based on a 2D still image codec, namely LAR (Locally adapted Resolution). It consists in a quad-tree representation of the images. The quad-tree representation contributes in the preservation of edges in both color and depth data. The adopted strategy is meant to be more perceptually driven than state-of-the-art methods. The proposed approach is compared to H.264 encoding of depth images. Objective metrics scores are similar with H.264 and with the proposed method, and visual quality of synthesized views is improved with the proposed approach

Hybrid tracking approach using optical flow and pose estimation

International audienceThis paper proposes an hybrid approach to estimate the 3D pose of an object. The integration of texture information based on image intensities in a more classical non-linear edge-based pose estimation computation has proven to highly increase the reliability of the tracker. We propose in this work to exploit the data provided by an optical flow algorithm for a similar purpose. The advantage of using the optical flow is that it does not require any a priori knowledge on the object appearance. The registration of 2D and 3D cues for monocular tracking is performed by a non linear minimization. Results obtained show that using optical flow enables to perform robust 3D hybrid tracking even without any texture mode

Focus on visual rendering quality through content-based depth map coding

International audienceMulti-view video plus depth (MVD) data is a set of multiple sequences capturing the same scene at different viewpoints, with their associated per-pixel depth value. Overcoming this large amount of data requires an effective coding framework. Yet, a simple but essential question refers to the means assessing the proposed coding methods. While the challenge in compression is the optimization of the rate-distortion ratio, a widely used objective metric to evaluate the distortion is the Peak-Signal-to-Noise-Ratio (PSNR), because of its simplicity and mathematically easiness to deal with such purposes. This paper points out the problem of reliability, concerning this metric, when estimating 3D video codec performances. We investigated the visual performances of two methods, namely H.264/MVC and Locally Adaptive Resolution (LAR) method, by encoding depth maps and reconstructing existing views from those degraded depth images. The experiments revealed that lower coding efficiency, in terms of PSNR, does not imply a lower rendering visual quality and that LAR method preserves the depth map properties correctly

BIT-RATE ALLOCATION BETWEEN TEXTURE AND DEPTH: INFLUENCE OF DATA SEQUENCE CHARACTERISTICS

International audienceThis paper questions the existence of factors influencing the quality of the synthesized views, in the context of multi-view video plus depth coding (MVC). The issue of bit-rate allocation between texture and depth data in MVC is still open, despite the many efforts already raised for the development of optimization techniques. The originality of this study lies in the investigation of direct relationships between the best bit-rate allocation, in terms of objective quality of synthesized views, and the sequence characteristics (entropy of depth maps, depth complexity and camera baseline distance, background/foreground contrast areas). The results confirm our assumptions regarding the impact of the sequence features on the bit-rate allocation. The results and the limitations of the study are also discussed

Real-time 3D model-based tracking: Combining edge and texture information

International audienceThis paper proposes a real-time, robust and efficient 3D model-based tracking algorithm. A non linear minimization approach is used to register 2D and 3D cues for monocular 3D tracking. The integration of texture information in a more classical non-linear edge-based pose computation highly increases the reliability of more conventional edge-based 3D tracker. Robustness is enforced by integrating a M-estimator into the minimization process via an iteratively re-weighted least squares implementation. The method presented in this paper has been validated on several video sequences as well as in visual servoing experiments considering various objects. Results show the method to be robust to large motions and textured environments

Fusion de primitives visuelles pour le suivi 3D temps-réel

National audienceCet article présente une méthode de suivi 3D basé modèle, précise et robuste. L'information apportée par le motif de l'objet est intégrée à un algorithme classique de calcul de pose reposant sur ses contours de façon à obtenir un résultat plus fiable. La robustesse du suivi est assurée par l'utilisation des M-estimateurs dans le processus d'estimation et un modèle multi-échelle des motifs. L'approche a été validée sur des séquences vidéos ainsi que sur des applications en ligne de Réalité Augmentée. Ces résultats montrent que la méthode proposée permet une localisation à la cadence vidéo tout en étant robuste aux larges mouvements et à des environnements réalistes

Optimized fixed point implementation of a local stereo matching algorithm onto C66x DSP

International audienceStereo matching techniques aim at reconstructing disparity maps from a pair of images. The use of stereo matching techniques in embedded systems is very challenging due to the complexity of the state-of-the-art algorithms. An efficient local stereo matching algorithm has been chosen from the literature and implemented on a c6678 DSP. Arithmetic simplifications such as approximation by piecewise linear functions and fixed point conversions are proposed. Thanks to factorisation and pre-computing, the memory footprint is reduced by a factor 13 to fit on the memory footprint available on embedded systems. A 14.5 fps speed (factor 60 speed-up) has been reached with a small quality loss on the final disparity map

Bit-rate allocation for Multi-view video plus depth

International audienceThe efficient compression of multi-view-video-plus-depth (MVD) data raises the bit-rate allocation issue for the compression of texture and depth data. This question has not been solved yet because not all surveys reckon on a shared framework. This paper studies the impact of bit-rate allocation for texture and depth data relying on the quality of an intermediate synthesized view. The results show that depending on the acquisition configuration, the synthesized views require a different ratio between the depth and texture bit-rate: between 40\% and 60\% of the total bit-rate should be allocated to depth

An edge-based structural distortion indicator for the quality assessment of 3D synthesized views

International audience3D-TV applications require the generation of novel viewpoints through Depth-Image-Based-Rendering methods. These synthesized views need to be assessed by a reliable quality metric. Most of the proposed metrics are inspired from 2D commonly used quality metrics. Yet, the latter were originally designed to address 2D compression distortions which are different from the distortions related to DIBR processes. We propose an edge-based method that indicates the level of structural degradation in the synthesized image. The first results are encouraging since the correlation to subjective scores is higher than other tested metrics

Modélisation de Scènes Naturelles à Partir de Séquences Vidéos Multi-vue plus Profondeur (MVD)

National audienceDans cet article, un schéma de modélisation de séquences Multi-vues Vidéo plus profondeur (MVD) est présenté. Le but est de réduire la redondance de profondeur et de texture présentes dans les séquences MVD. Pour ce faire, la fusion de cartes de profondeurs utilisant une représentation volumétrique est proposée. Les voxels sont "carvés" itérativement pour chaque vue en utilisant la technique de traçage de rayons (ray tracing). La surface fusionnée est extraite à partir de cette représentation en utilisant l'algorithme de Marching Cubes. Finalement, le problème de plaquage des textures sur cette surface résultante est abordé. L'algorithme proposé sélectionne parmi toutes les textures le meilleur candidat pour texturer un triangle de la surface résultante. Ce choix est fait en utilisant une métrique dite de photocohérence. Les tests et les résultats sont fournis pour des images fixes en utilisant les séquences MVD usuelles